Hierarchical 2-D mesh representation, tracking, and compression for object-based video

This paper proposes methods for designing, tracking and coding hierarchical 2D content-based mesh representations. The design procedure consists of constructing a ne-to-coarse hierarchy of Delaunay meshes, using image-and shape-based criteria for mesh geometry simpliication. Hierarchical tracking employs a coarse-tone strategy with mesh-based motion vector optimization. We introduce new techniques to maintain the initial mesh hierarchy and topology during tracking by imposing certain constraints at each stage of the procedure. The hierarchical compression technique is based on a nearest-neighbor ordering of mesh node points. This ordering serves to identify the mesh boundary nodes as well as establishing spatial predictors for diierential coding of node coordinates and motion vectors. The proposed hierarchical mesh representation, which has applications in object-based video manipulation, indexing, and compression, provides improved tracking performance (compared to a non-hierarchical representation), and allows progressive (scalable) transmission of the object geometry (including shape) and motion information as well as variable level-of-detail rendering. Experimental results are presented to compare the tracking and compression performance of hierarchical versus non-hierarchical mesh representations, and to demonstrate the trade-oo between image quality and mesh bit rate for 2D mesh-based video object rendering.